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any  way  supplied  the  said  drawings,  specifications,  or  other  data,  is  not  to  be 
regarded  by  implication,  or  otherwise  in  any  manner  construed,  as  licensing  the 
holder,  or  any  other  person  or  corporation,  or  as  conveying  any  rights  or 
permission  to  manufacture,  use,  or  sell  any  patented  invention  that  may  in  any 
way  be  related  thereto. 

The  Public  Affairs  Office  has  reviewed  this  report  and  it  is  releasable  to  the 
National  Technical  Information  Service,  where  if  will  be  available  to  the  general 
public,  including  foreign  nationals. 
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PREFACE 


This  research  effort  described  by  this  paper  was  performed  under  Armstrong  Laboratory 
Logistics  Research  Division  Work  Unit  1710-00-40,  Task  32,  Modeling  the  Requirements 
Process,  through  a  contract  (F33615-88-C-0004)  to  Systems  Exploration,  Inc.  Mr.  Art 
Schwaninger  was  the  principal  investigator.  He  was  assisted  by  Mr.  Mark  Miller,  Ms.  Patti  Jo 
Vore,  and  Ms.  Colleen  Gumienny. 

This  paper  is  a  product  of  a  study  effort  aimed  at  gaining  an  in-depth  understanding  of 
the  Air  Force  major  weapon  system  requirements  process  as  it  is  described  in  the  various 
governing  directives,  and  an  understanding  of  how  those  directives  are  interpreted  and  practiced 
in  the  field.  The  study  examined  the  role  of  computer  support  for  the  process;  this  paper 
describes  the  development  of  a  graphic  description  of  the  requirements  process.  Integrated 
Computer-Aided  Manufacturing  Definition  methods  were  the  media  for  this  description. 

The  authors  wish  to  thank  Captain  Raymond  R.  Hill  for  his  assistance  and  guidance, 
especially  in  the  area  of  study  design,  and  Mr.  Brett  Andrews  of  the  Air  Force  Institute  of 
Technology  School  of  Systems  and  Logistics  for  his  assistance  in  the  development  of  the 
structured  interviews  with  subject  matter  experts.  We  also  wish  to  thank  the  many  subject 
matter  experts  who  provided  insight  into  both  the  interpretation  of  directives  and  the  practice  of 
managing  evolving  requirements. 
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SUMMARY 


This  paper  describes  the  environment  associated  with  the  identification,  validation, 
tracking,  and  management  of  operational  and  support  requirements  for  Acquisition  Category 
(A CAT)  I  weapon  systems  within  the  Air  Force,  including  the  processes  and  subprocesses, 
products  and/or  documentation  required  and  produced,  and  the  organizations  involved.  It  also 
identifies  and  describes  opportunities  for  the  fabrication  and  use  of  a  decision  support  system 
(DSS)  to  improve  various  subprocesses. 

The  description  and  analysis  of  the  requirements  environment  was  facilitated  by  the 
construction  of  Integrated  Computer-Aided  Manufacturing  Definition  (IDEF)  model  descriptions 
of  the  requirements  process.  (Integrated  Computer-Aided  Manufacturing  Definition,  Activity 
Modeling  (IDEFO)  and  Integrated  Computer-Aided  Manufacturing  Definition,  Process  Modeling 
(IDEF3)  were  the  specific  methods  used.)  The  requirements  process  as  defined  for  this  paper, 
extends  from  the  need  determination  or  pre-Milestone  0  phase  through  the  Milestone  ID  decision 
to  enter  into  the  production  and  deployment  of  a  weapon  system. 

The  source  of  the  data  used  to  construct  the  models  came  from  three  primary  sources: 

(a)  the  government  directives:  Department  of  Defense  Directive  (DoDD)  5000.1,  Department  of 
Defense  Instruction  (DoDI)  5(X)0.2,  and  Department  of  Defense  Manual  (DoDM)  5000.2-M,  and 
Air  Force  Regulation  (AFR)  57-1;  (b)  interviews  with  subject  matter  experts  from  the  operating 
commands  (Headquarters  (HQ),  Tactical  Air  Command  (TAC),  and  Strategic  Air  Command 
(SAC)),  implementing  command  (HQ  Air  Force  Systems  Command  (AFSC)  Aeronautic 
Systems  Division  (ASD)),  supporting  command  (HQ  Air  Force  Logistics  Command  (AFLC), 
and  Oklahoma  City  Air  Logistics  Center),  Air  Staff  (primarily  the  Policy  and  Joint 
Requirements  Division  (AF/XORJ)  and  the  office  of  the  Deputy  Assistant  Secretary 
(Management  Policy  and  Program  Integration)  (SAF/AQX)),  and  faculty  members  of  the  Air 
Force  Institute  of  Technology  (AFIT)  School  of  Systems  and  Logistics;  and  (c)  an  extensive 
library  of  current  literature  provided  by  Armstrong  Laboratory. 

The  research  performed  under  this  contract  indicates  that  a  DSS  would  greatly  enhance 
the  Mission  Area  Assessment  (MAA)  and  Mission  Need  Analysis  (MNA)  segments  of  the  need 
determination  process.  Furthermore,  a  DSS  would  emphasize  and  encourage  the  employment  of 
concurrent  engineering  (CE)  principles  throughout  the  acquisition  process.  It  would  be  of 
considerable  value  in  integrating  the  vast  amounts  of  similar  data  in  the  myriad  documents 
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involved  in  an  ACAT I  acquisition.  Finally,  the  DSS  can  be  readily  adapted  to  fulfill  the 
pressing  need  for  training  of  personnel  working  within  the  requirements  process 


Based  on  the  research  conducted  under  this  contract,  it  is  recommended  that  a  DSS  be 
constructed  for  ACAT  III  and  lesser  acquisitions,  and  that  research  begin  to  identify  existing 
software  packages  that  could  be  readily  adapted  to  such  a  DSS.  A  DSS  that  could  significantly 
enhance  the  requirements  process  is  well  within  current  technology 

L  INTRODUCTION 
Purpose 

The  purpose  of  this  effort  was  to  research  the  processes  by  which  Air  Force  weapon 
system  requirements  are  generated,  identified,  validated,  traded,  documented,  tracked,  and 
managed  within  the  larger  acquisition  process.  The  results  of  this  research  were  then  used  to 
identify  those  activities  or  subprocesses  that  could  be  improved  through  the  application  of 
computerized  decision  support  technology. 

The  scope  of  this  task  extends  from  the  Determination  of  Mission  Need  (the  Pre-Milestone 
0  Phase)  through  the  Milestone  III  Production  Approval  decision.  The  scope  includes  only  major 
acquisitions  that  are  designated  ACAT  I. 

Modeling  provided  a  structured  means  to  gain  an  in-depth  understanding  of  the 
environment  of  Air  Force  requirements  and  acquisition  personnel,  and  to  describe  the  functions 
and  processes  of  system  acquisition  as  they  relate  to  mission  and  system  requirements. 

Two  sets  of  models  were  constructed.  One  set  describes  the  processes  prescribed  by 
Department  of  Defense  (DoD)  directives;  the  other  describes  the  practical  interpretation  and 
application  of  these  directives  by  field  personnel.  After  the  models  had  been  developed  and 
refined,  they  served  as  the  foundation  for  an  analysis  of  decision  support  technology 
opportunities. 


Report  Structure 

This  report  summarizes  the  technical  work  accomplished  and  the  information  gained  by 
modeling  the  requirements  process.  The  report  contains  a  discussion  of  the  background  factors 
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which  led  to  the  need  for  this  research,  including  a  sununary  of  some  of  the  major  changes  to  the 
requircments  process,  followed  by  a  chronology  of  the  task.  This  chronology  includes  a 
complete  discussion  of  the  rationale,  methodologies  employed,  and  basic  assumptions  of  the 
research  approach.  The  report  then  focuses  on  the  IDEF0/IDEF3  modeling  task,  discusses 
relevant  acquisition  topics,  and  presents  conclusions  and  recommendations  for  future  studies. 

n.  BACKGROUND 
Defense  Acquisition  Reform 

Defense  spending  and  acquisition  practices  generated  considerable  public  concern  in  the 
mid-1980s.  During  this  period,  the  media  was  fraught  with  stories  of  overpriced  spare  parts,  test 
deficiencies,  and  cost  and  schedule  overruns.  Consequently,  the  President  appointed  a  Blue 
Ribbon  Commission  on  Defense  Management  (often  referred  to  as  the  Packard  Commission)  to 
evaluate  the  defense  acquisition  system  and  determine  how  it  could  be  improved.  The 
Commission  sought  changes  that  would  lower  the  cost  of  military  equipment  without 
compromising  performance. 

The  Commission  analyzed  repons  of  gross  inefficiencies  and  determined  that  the  "cure" 
would  require  more  than  bandages  applied  to  isolated  surface  problems.  After  concluding  its 
efforts,  the  Commission  produced  a  host  of  recommendations  designed  to  address  significant, 
structurally  embedded  problems,  such  as  the  length  of  the  acquisition  cycle. 

In  addition  to  the  Packard  Commission,  other  critics  with  both  rank  and  experience  in  the 
process  have  written  extensively  on  the  ills  of  the  system  as  practiced  before  the  release  of  the 
DoD  5000  series  of  directives  and  instructions  in  February  1991.  Two  notable  critics  are 
Lt.  General  Thomas  R.  Ferguson  and  retired  Lt.  General  Glenn  A.  Kent.  General  Ferguson 
served  as  principal  deputy  within  the  Office  of  the  Assistant  Secretary  of  the  Air  Force  for 
Acquisition  and  as  Commander  of  the  Air  Force  Materiel  Command,  Aeronautical  Systems 
Center,  the  largest  of  the  Air  Force  acquisition  centers.  General  Kent,  a  senior  analyst  with 
RAND  Corporation,  at  one  time  directed  the  Air  Force  Studies  and  Analysis  Group. 

The  possibility  that  the  recommendations  of  the  Packard  Commission  and  other  defense 
critics  might  result  in  the  release  of  new  DoD  policies  and  procedures  in  the  midst  of  our  study 
was  acknowledged  at  the  inception  of  this  effort.  Sweeping  changes  were  certain  to  affect  both 
the  "prescribed"  and  "actual  practice"  models.  In  fact,  the  release  of  DoDD  50(X).l,  DoDI 
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5000.2,  and  DoDM  5000.2M  op  23  February  1991,  roughly  one  month  after  this  effort  began, 
had  a  major  influence  on  th(.  ourse  of  the  research. 

A  primary  outcome  of  the  defense  acquisition  reform  initiated  by  the  recommendations 
of  the  Packard  Commission  was  the  establishment  of  a  set  of  management  offices:  an  office 
within  t'.e  Secretary  of  Defense,  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition; 
an  office  within  the  Air  Force,  the  Office  of  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition)  (SAF/AQ);  and,  more  recently,  the  creation  of  the  Air  Force  Program  Executive 
Office  (AFPEO).  SAF/AQ  commands  the  AFPEO  that  consists  of  the  Assistant  Secretary,  his 
principal  deputy  assistant  secretary,  and  a  cadre  of  seven  program  executive  officers  (PEOs). 

The  AFPEO  is  responsible  for  shaping  and  executing  weapon  system  development  and 
production  plans. 

Total  Quality  Management  and  Concurrent  Engineering 

In  addition  to  the  new  acquisition  directives  and  reorganizations,  several  other  major 
initiatives  are  ongoing  within  the  DoD,  such  as  the  commitment  to  Total  Quality  Management 
(TQM).  TQM  initiatives  began  primarily  in  response  to  international  competitive  challenges  to 
the  American  industrial  base.  The  DoD  posture  on  quality  issued  by  the  Secretary  of  Defense, 
Frank  Carlucci,  in  March  1988,  gave  top  priority  to  the  TQM  effort  as  a  vehicle  for  attaining 
continuous  quality  improvement  in  Air  Force  operations  and  as  a  major  strategy  to  meet  the 
President's  productivity  objectives  under  Executive  Order  12552. 

In  an  address  to  the  Fourth  Quality  Improvement  Symposium  at  Wright-Patterson  Air 
Force  Base  in  September  of  1989,  then  Under  Secretary  of  Defense  for  Acquisition,  John  A. 
Betti,  stated,  "The  best  way  to  enhance  DoD-industry  relations  is  by  improving  the  quality  of 
defense  products  and  services."  At  the  very  heart  of  this  is  the  DoD  TQM  initiative. 

A  critical  component  of  TQM  is  CE,  which  recognizes  the  need  for  concurrent  product 
and  process  design.  CE,  which  is  synonymous  with  other  terms  such  as  concurrent  design, 
simultaneous  engineering,  and  integrated  product  development,  integrates  the  product 
engineering  process  with  the  product  manufacturing  processes  and  the  product  support 
processes,  emphasizing  efficiency,  increased  quality,  and  reduced  cost. 

The  goals  of  TQM  and  CE  mesh  extremely  well  with  a  new  management  approach  called 
Quality  Function  Deployment  (QFD).  QFD  has  been  successfully  implemented  in  the  Japanese 


auto  industry,  resulting  in  dramatic  reductions  in  start-up  and  pre-production  costs  at  Toyota. 
QFD  has  been  called  "simplified  systems  engineering"  and  "the  voice  of  the  customer."  It 
provides  a  vehicle  to  enhance  requirements  definition  and  understanding.  QFD  also  provides 
traceability,  and  correlates  and  integrates  requirements-associated  data  currently  spread  among  a 
multitude  of  acquisition  documents. 

The  QFD  technique  encompasses  a  matrix  structure  and  group  analysis  approach,  thus 
providing  a  method  to  conceptually  map  the  views  of  multiple  functions.  The  QFD  technique 
also  serves  as  a  means  for  inter-functional  planning  and  communication. 

The  change  in  managerial  direction  and  organization  spurred  the  adoption  of  TQM  and 
QFD  techniques,  and  created  a  malleable  environment  ripe  for  the  introduction  of  decision 
support  technologies  to  support  the  weapon  systems  requirements  process. 

Summary  of  Changes  to  the  Requirements  Process. 

The  23  February  1991  release  of  the  primary  DoD  guidance  on  the  requirements/ 
acquisition  process  includes  DoDD  5000.1,  DoDI  5000.2,  and  DoDM  5000.2M.  These 
documents  ushered  in  a  new  philosophy  based  on  streamlining  the  acquisition  process  (a  major 
recommendation  of  the  President's  Blue  Ribbon  Commission,  resulting  in  61  documents  being 
either  canceled  and/or  consolidated  into  the  three  "new"  5000  series  documents)  and  focusing  on 
better  requirements  determination/definition  at  the  outset  of  the  process.  The  latter  clearly 
reflecting  General  Kent's  caution  that  a  "requirements"  exist  for  operational  capabilities,  not  for 
specific  hardware  and  software  solutions.  In  other  words,  "requirements"  are  derived  from  the 
National  Security  Objectives  and  weapon  system  performance  characteristics  are  merely  the 
means  by  which  the  "requirements"  are  met. 

While  the  changes  in  the  DoD  acquisition  process  are  many  and  a  detailed  explanation  of 
them  all  would  exceed  the  scope  of  this  paper,  the  basic  framework  of  the  process,  referring  to 
the  concept  of  a  graduating  sequence  of  milestone  decisions  based  on  the  successful  completion 
of  each  preceding  phase,  is  still  intact.  Table  1  provides  a  comparison  of  the  old  versus  the  new 
program  milestones  and  phases.  The  primary  differences  occur  at  the  beginning  of  the  process. 
The  most  striking  difference  is  that  program  initiation,  the  establishment  of  a  system- 
specific/programmatic  organization  (usually  termed  a  System  Program  Office  (SPO)),  does  not 
occur  in  the  new  scheme  until  after  successful  completion  of  the  Concept  Exploration  and 
Definition  Phase. 
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Table  1.  Comparison  of  Program  Milestone  and  Phases 


QLn 

1.  Milestone  0,  Program  Initiation/Mission 
Need  Decision. 

2.  Phase  I,  Concept  Exploration/Definition. 

3.  Milestone  I,  Concept  Demonstration/ 

Validation  Decision. 

4.  Phase  II,  Concept  Demonstration/Validation. 

5.  Milestone  n,  Full-Scale  Development  Decision. 

6.  Phase  III,  Full-Scale  Development/ 
Manufacturing. 

7.  Milestone  III,  Full-Rate  Production  and 
Initial  Deployment. 

8.  Phase  IV,  Full-Rate  Production  and  Initial 
Deployment. 

9.  Phase  V,  Operations  and  Support 
(overlaps  Phase  IV). 

10.  Milestone  IV,  Logistics  Readiness  and 
Approval. 

11.  Milestone  V,  Major  Upgrade  or  System 
Replacement  Decision. 


NEW 

1.  Milestone  0,  Concept  Studies 
Approval. 

2.  Phase  0,  Concept  Exploration/ 
Definition. 

3.  Milestone  I,  Concept  Demonstration 
Approval.  Program  Initiation. 

4.  Phase  I,  Demonstration  &  Validation. 

5.  Milestone  II,  Development  Approval. 

6.  Phase  II,  Engineering  and 
Development. 

7.  Milestone  III,  Production  Approval. 

8.  Phase  III,  Production  and 
Deployment. 

9.  Phase  IV,  Operations  and  Support 
(overlaps  Phase  III). 

10.  Milestone  IV,  Major  Modification 
Support  Review. 

1 1.  No  Milestone  V  in  new 
DoDD  5000.2. 


Other  significant  changes  occurred  in  Air  Force  and  milestone  review  documentation 
requirements.  Table  2  compares  the  old  and  new  documentation  requirements  by  milestone  as 
well  as  terminology.  Although  there  may  be  subtle  differences,  an  obvious  correlation  exists 
between  the  "old"  and  the  "new"  documentation  (e.g.,  an  "old"  Statement  of  Operational  Need 
(SON)  roughly  equates  to  a  "new"  Mission  Need  Statement  (MNS),  an  "old"  System  Operational 
Requirements  Document  (SORD)  is  basically  the  same  as  a  "new”  Operational  Requirements 
Document  (ORD),  etc.) 
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Table  2.  Comparison  of  Air  Force  and  Milestone  Review  Documents 

(ACAT I  Programs) 


OLD 


mm 


Milestone  0 

1.  SON. 

2.  Cooperative  Opportunities  Document  (COD). 

3.  Independent  Cost  Estimate  (ICE). 

Milestone  I 

1.  SORD. 

2.  Depot  Support  Requirements  Document. 

3.  System  Concept  Paper. 


4.  Baseline  Correlation  Matrix. 

5.  COD. 

6.  Competitive  Prototyping  Strategic  Waiver. 

7.  System  Threat  Assessment  Report  (STAR). 

8.  Test  &  Evaluation  Master  Plan  (TEMP). 

9.  ICE. 

10.  Program  Baseline. 

11.  Ctost  and  Operational  Effectiveness  Analysis 
(COEA). 


1.  MNS. 

2.  COD  not  Required  for  Milestone  0. 

3.  ICE  not  Required  for  Milestone  0. 


1.  Replaced  by  ORD. 

2.  Replaced  by  ORD. 

3.  Integrated  Program  Summary  (IPS). 
Annex  A:  Program  Structure. 

Annex  B:  Program  Life  Cycle  Cost 

Estimate  Summary. 

Annex  C:  Acquisition  Strategy  Report. 
Annex  D:  Risk  Assessment. 

Annex  E:  Environmental  Analysis. 
Annex  F:  Affordability  Assessment. 
Annex  G:  Cooperative  Opportunities 
Document. 

4.  Replaced  by  System  Maturity  Matrix 
(SMM). 

5.  Incorporated  into  Annex  G,  IPS. 

6.  Not  Specifically  Required,  Contained  in 
Acquisition  Strategy  (Annex  C,  IPS). 

7.  STAR. 

8.  TEMP. 

9.  ICE. 

10.  Concept  Baseline. 

11.  COEA. 
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Table  2.  Comparison  of  Air  Force  and  Milestone  Review  Documents  (Continued) 


1 

j 


Mitotapg  I 

12.  Joint  Requirements  Oversight  Council 
(JROC)  Assessment. 

13.  Integrated  Program  Assessment. 

14.  Program  Life  Cycle  Cost  Estimate . 

Milestone  II 

Same  as  Milestone  I  except; 

1.  Decision  Coordinating  Paper. 

2.  Manpower  Estimate  Report  (MER). 

3.  Acquisition  Strategy  Report. 

4.  Program  Baseline. 

5.  Beyond  Low  Rate  Initial  Production 
(LRIP)  Report. 

Milestone  III 

Same  as  Milestone  II  except: 

1.  Beyond  LRIP  Report. 

As  previously  noted,  the  most  significant  changes  are  those  that  affect  the  early  stages  of 
requirements  definition.  The  emphasis  during  this  time  has  shifted  from  program 
management/financial  issues  to  user  operational/mission  requirement  issues.  The  operating 
command  has  been  given  lead  responsibility  (versus  the  implementing  command)  in  managing 
both  the  conduct  of  the  concept  studies  and  the  development  of  the  COEA. 

Both  concept  studies  and  COEAs  are  crucial  to  system  requirements  definition  because 
they  constitute  most  of  the  activity  during  the  Concept  Exploration  and  Definition  Phase 
(Phase  0).  To  facilitate  cooperation  and  teamwork  during  this  critical  pre-Milestone  I  phase,  the 
Air  Force  established  a  management  body  known  as  the  Concept  Action  Group  (CAG).  The 
operating  command  chairs  the  CAG  and  determines  CAG  membership  commensurate  with  the 


1.  Replaced  by  IPS. 

2.  MER. 

3.  Incorporated  into  Annex  C,  IPS. 

4.  Development  Baseline. 

5.  Not  Required  until  Milestone  III. 
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needs  of  the  particular  acquisition.  CAG  membership  could  include  Air  Staff,  implementing  and 
supporting  commands,  the  Air  Force  Operational  Test  and  Evaluation  Center  (AFOTEC),  and 
other  research,  study,  and  analysis  organizations  as  deemed  necessary. 

In  addition  to  the  CAG,  the  Air  Force  established  summit  requirements  and  acquisition 
program  reviews.  A  summit  is  a  senior-level  review  chaired  by  the  Air  Force  Chief  of  Staff 
(CSAF)  for  all  ongoing  major  defense  acquisition  programs.  A  summit  is  normally  convened 
once  during  each  phase,  prior  to  milestone  decisions,  to  review  and  affirm  user-stated  needs  and 
requirements.  The  summit  also  affirms  the  adequacy  of  program  development  efforts  to  satisfy 
those  needs  in  a  timely,  cost-effective  manner. 

Air  Force  Restructuring 

The  total  impact  of  Air  Force  restructuring  is  beyond  the  scope  of  this  paper.  The  Air 
Force  is  undergoing  an  upheaval  unprecedented  in  its  46  years  of  existence.  In  addition  to 
organizational  changes,  the  Air  Force  will  lose  about  22  percent  of  its  manpower  over  the  next 
several  years.  On  30  September  1990,  Air  Force  strength  stood  at  532,000.  The  current 
authorization  bill  calls  for  a  drop  to  510,000  by  the  end  of  fiscal  year  1991  (FY91)  and  to 
415,000  bytheendofFY95. 

Knowledgeable  insiders  predict  that  the  Air  Force  will  reduce  its  management  structure 
in  the  major  commands  by  over  30  percent  and  that  Air  Staff  will  undergo  a  similar  reduction  by 
up  to  30  percent.  The  impact  of  international  events  and  internal  pressures  cannot  be  understated 
and  are  bound  to  result  in  a  fundamental  reshaping  of  the  military  services. 

Restructuring  is  occurring  along  two  fronts:  operations  and  acquisition/logistics.  Within 
the  operations  community,  TAG,  SAC,  and  the  Military  Airlift  Command  (MAC)  have  been 
reorganized  into  two  commands.  SAC  and  TAC,  with  elements  of  intratheater  airlift-type 
aircraft  from  MAC  have  merged  to  form  the  Air  Combat  Command.  The  new  command 
operates  self-sufficient  "composite  wings"  composed  of  the  various  types  of  aircraft  needed  for 
foreseeable  combat  scenarios.  Thus,  these  composite  wings  contain  fighters,  bombers, 
reconnaissance  aircraft,  tactical  airlift,  and  tanker  aircraft.  The  concept  has  been  described  as 
"One  wing,  one  base,  one  boss"  and  presumably  "one  headquarters."  The  Air  Mobility 
Command,  composed  of  the  remaining  MAC  functions,  provides  strategic  airlift  capability. 
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In  the  acquisition/logistics  arena,  AFLC  and  AFSC  have  merged  to  form  the  Air  Force 
Materiel  Command  (AFMC).  As  with  the  operating  forces,  this  is  a  tremendous  cultural  change 
for  command  members.  Over  the  years,  the  AFLC  and  AFSC  have  operated  with  distinctly 
different  cultures,  a  natural  result  of  radically  different  missions,  based  on  the  between  the 
acquisition  and  logistics  communities.  Recent  streamlining  of  both  commands  coupled  with  the 
quality  revolution  are  the  reasons  combining  now  appears  feasible. 

The  concept  underlying  the  new  command  is  Integrated  Weapon  System  Management 
(IWSM).  IWSM  recognizes  the  need  for  acquisition  and  logistics  systems  to  work  together  from 
the  very  beginning— a  "cradle-to-grave"  approach  to  materiel  management.  An  advantage  of  this 
concept  is  that  it  presents  a  single  point  of  contact  and  a  consistent  contracting  style  to  both  the 
using  command  and  the  defense  industry.  The  intent  of  the  new  command  is  to  present  one  face 
to  industry;  IWSM,  creates  total  visibility  over  a  weapon  system  with  one  program  director 
supported  by  labs,  acquisition  and  logistics  professionals. 

Impact  of  Restructuring  on  Project  Analysis 

Given  the  degree  of  organizational  turbulence  within  the  Air  Force,  the  impact  of  these 
changes  on  the  "requirements"  process  is  impossible  to  predict  precisely.  As  operating, 
implementing,  and  supporting  commands  merge  and  missions  change,  it  is  difficult  to  identify 
the  command  whose  previous  methods  will  prevail  or  determine  whether  the  new  procedures 
will  demand  a  totally  new  way  of  doing  things.  The  latter  seems  probable;  however,  time  must 
pass  before  the  new  procedures  are  fully  defined.  Some  of  the  initial  and  untested  procedures 
will  likely  require  modification.  This  hypothesis  is  supported  by  the  numerous  revisions  to 
AFR  57-1.  The  most  recent  revision  is  expected  to  be  a  cancellation  of  the  8  November  1991 
version  in  order  to  issue  policy  and  a  supporting  pamphlet. 

The  products  of  this  research  (the  models  and  description  of  the  requirements  process) 
reflect  only  the  directives  in  force  and  the  subject  matter  expert  (SME)  knowledge  during  the 
data  gathering  period  of  this  effort  May  to  November  1991. 

Decision  Support  System  for  Requirements 

The  Air  Force  commitment  to  the  principles  of  TQM  and  CE,  in  an  environment  of 
acquisition  reform,  has  spawned  additional  research  to  improve  the  tools  and  techniques  applied 
within  the  weapon  system  acquisition  process.  Specifically,  this  research  is  targeted  toward 
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definition,  development,  and  demonstration  of  computer-based  tools  and  supporting 
methodologies  that  have  potential  to  enhance  definition  and  design  efforts  early  in  the  weapon 
system  acquisition  process. 

Our  research  suggests  that  such  tools  can  be  utilized  long  before  an  acquisition  contract  is 
awarded.  For  example,  DSS  technology  could  be  used  by  operators  to  help  formulate  and 
prioritize  mission  requirements  within  a  strategies-to-task  framework.  Mission  requirements, 
expressed  in  this  manner  by  the  ultimate  user,  are  frequently  referred  to  as  the  "voice  of  the 
customer." 

DSS  technology  can  provide  sequential  support  to  users:  first,  by  helping  a  user 
determine  an  operational  shortfall;  next,  by  assisting  in  the  evaluation  of  present  equipment.  If 
the  new  requirement  cannot  be  met  by  anything  other  than  materiel  acquisition  or  modification, 
the  DSS  assists  in  alternative  evaluation  and  selection.  In  this  manner,  DSS  technology  can 
guide  successful  management  decisions  for  the  design,  development,  and  production  of  an  item. 
These  decisions  reflect  a  solid  foundation  of  mission  requirements  that  are  explicitly  related  to 
national  security  objectives.  Arbitrary  decision-making  is  replaced  by  rigorous  design  analysis. 
In  fact,  close  and  explicit  coupling  of  design  requirements  to  mission  requirements  reduces  the 
risk  of  encountering  an  unplanned  design  shortfall. 


m.  TASK  CHRONOLOGY 

This  effort  was  organized  into  four  logical  phases.  The  phase  objectives  and 
methodologies  for  each  are  described  in  the  following  paragraphs. 

Phase  I:  Preparation 

Phase  I,  Preparation,  comprised  three  objectives:  of  this  phase  were  to  (a)  develop, 
approve,  and  deliver  a  detailed  research  plan  (DRP);  (b)  initiate  the  literature  search, 
concentrating  on  the  DoD/Air  Force  guidance/directives,  and  (c)  receive  instruction  in  the 
IDEF3  process  modeling/description  capture  methodology  (IDEF3)  and  undertake 
familiarization  training  on  prototype  IDEF3  modeling  software. 
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Detailed  Research  Plan 


The  DRP  reflected  the  Statement  of  Work  (SOW)  tasking  to  model  the  requirements 
process  from  two  perspectives:  (a)  the  process  as  prescribed  by  governing  directives  and  (b)  the 
process  as  actually  practiced  by  field  personnel.  Modeling  was  performed  from  two  perspectives 
because  the  differences  between  the  two  would  indicate  candidate  areas  for  DSS  applications. 

Another  consideration  for  the  DRP  was  the  SOW  requirement  to  express  the  findings  of 
this  effort  via  two  related  IDEF  techniques:  Activity  Modeling  (IDEFO)  and  Process  Modeling 
(IDEF3).  This  requirement  necessitated  the  construction  of  eight  (two  schoolhouse  and  six 
real-world)  models.  Information  to  construct  the  models  was  taken  from  both  the  literature 
search  and  interviews  with  acquisition  personnel  in  the  various  organizations. 

IDEFO  activity  modeling  captures  relationships  of  elements  without  regard  to  temporal 
relationships.  IDEF3  process  modeling  extends  the  relationships  identified  through  IDEFO 
modeling  to  include  temporal  relationships.  By  virtue  of  this  connection,  the  pair  of  IDEF 
techniques  fully  describes  the  requirements  process. 

Since  the  prescribed  process  is  the  same  perspective  taught  by  the  AFIT  School  of 
Systems  and  Logistics,  and  since  our  information  collection  instrument  was  to  be  reviewed  by 
AFIT  faculty,  this  model  was  identified  as  the  "Schoolhouse"  version.  For  research  planning 
purposes,  the  "Schoolhouse"  model  was  assumed  to  include  all  activities  of  all  Air  Force 
organizations.  In  IDEF  terminology,  this  model  reflects  an  "objective"  viewpoint.  Thus,  it  does 
not  present  any  particular  parochial,  command,  or  organizational  slant.  Conversely,  the  second 
model  is  the  "Practice"  version.  The  perspective  of  this  model  considers  three  viewpoints:  SAC, 
TAC,  and  the  ASD  of  AFSC). 

Assumptions  in  the  DRP  include  the  availability  of  the  Knowledge  Based  Systems 
Laboratory  (KBSL)  prototype  IDEF3  software  and  either  an  in-house  host  or  the  unrestricted  use 
of  the  Armstrong  Laboratory,  Logistics  Research  Division  (AL/HRG)  computer  lab  to  build  the 
IDEF3  models. 

The  DRP  addresses  the  collection  of  adequate  source  data.  A  literature  search  and  data 
collection  interviews  with  acquisition  personnel  representing  various  Air  Force  organizations 
were  the  primary  data  gathering  methods  used  for  this  effort. 
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The  DRP  assumes  that  the  AL/HRG  Task  Monitor  will  secure  the  cooperation  of  the 
SMEs’  organizations  and  arrange  an  interview  schedule  convenient  to  the  SMEs  and  consonant 
with  the  task  schedule. 

A  recognized  element  of  risk  was  the  immature  nature  of  the  IDEF3  methodology  and 
associated  IDEF3  prototype  software.  Consequently,  it  was  determined  that  the  tool  should  not 
cost  more  in  man-hours  than  it  is  worth.  That  is,  if  bugs  in  the  software  caused  an  excessive 
man-hour  drain,  an  alternate  method  would  be  instituted,  including  hand  drawings,  if  necessary. 

The  DRP  further  acknowledges  the  risk  that  the  release  of  new  DoD  policies  might 
dramatically  alter  the  "schoolhouse"  version.  This  risk  in  conjunction  with  incremental  funding 
policies,  jeopardized  the  work  flow  of  this  effort. 

Literature  Search 

Many  relevant  documents  were  identified  through  the  literature  search.  The  timely 
topics  of  acquisition  reform  and  requirements  improvement  were  thoroughly  addressed  by 
technical  journals,  monographs,  and  policy  directives.  Combined,  these  sources  provided  ample 
data  to  support  the  modeling  and  analysis  effort. 

The  rapid  pace  of  change  during  the  time  frame  of  our  effort,  made  modeling  the 
schoolhouse  or  prescribed  process,  as  defined  in  the  governing  directives,  extremely  difficult. 

For  example,  shortly  after  task  initiation,  new  DoD  policy  was  issued  in  the  form  of 
DoDD  5000.1  (Defense  Acquisition),  and  its  supporting  publications,  DoDI  5000.2  (Defense 
Acquisition  Management  Policies  and  Procedures)  and  DoDM  5000.2M  (Defense  Acquisition 
Management  Documentation  and  Reports).  The  new  policy  was  a  major  revision;  it  responded 
to  the  criticism  and  recommendations  of  the  Blue  Ribbon  Commission  and  incorporated  many  of 
Ferguson  and  Kent's  views. 

Thus  this  effort  was  undertaken  at  a  time  when  the  Air  Force,  including  its  major 
commands,  had  not  yet  had  sufficient  opportunity  to  publish  service-level  and  command-specific 
directives.  This  lack  of  guidance  was  detrimental  because  these  directives  provide  implementing 
direction  for  daily  business  activities. 
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At  the  time  the  DoD  5000  series  was  released,  the  primary  Air  Force  directive  was  AFR 
57-1  (Air  Force  Mission  Needs  and  Operational  Requirements  Process)  dated  7  October  1988. 
This  version  was  incompatible  with  the  newly  specified  DoD  direction.  Subsequently,  a  draft 
AFR  57-1  was  published  on  2  April  1991.  Because  of  the  magnitude  of  the  DoD  changes  and 
the  many  comments  raised  during  the  coordination  cycle,  a  final  version  of  AFR  57-1  was  not 
published  until  8  November  1991.  Additionally,  an  Air  Force  supplement  to  DoD  5000.2  was 
being  compiled;  this  further  frustrated  any  attempts  to  model  business  activities. 

Additionally,  the  800  series  of  regulations  pertaining  to  Acquisition  Management  was 
pronounced  "unusable  in  its  present  form"  by  the  Pentagon  staff  officer  charged  with  rewriting 
and  maintaining  them.  At  the  time  of  the  interview  with  this  officer,  an  anticipated  publication 
date  for  revised  Air  Force  direction  was  unknown. 

The  extent  to  which  these  changes  complicated  this  effort  cannot  be  understated.  In 
short,  what  was  to  be  a  stable  foundation  of  our  analysis  (the  prescribed  requirements 
procedures)  proved  to  be  dynamic.  This  impaired  our  ability  to  model,  and  also  devalued  the 
historical  expertise  of  the  SMEs  scheduled  to  be  interviewed  as  sources  of  valid  data.  The  SMEs 
were  understandably  unfamiliar  with  the  business  and  procedural  implications  of  the  new 
procedures,  especially  in  the  area  of  ACAT I  (i.e.,  systems  with  research,  development,  test  and 
evaluation  costs  exceeding  $300  million  or  with  total  expenditures  expected  to  exceed  $1.8 
billion  in  1990  constant  dollars).  Since  no  acquisitions  which  meet  these  criteria  were  processed 
during  this  task  many  of  the  revisions  had  not  yet  been  circulated  for  comment,  it  was  quite 
likely  that  field  personnel  would  be  unable  to  anticipate  the  effect  of  new  directives. 

The  new  procedures  also  obscured  the  subtle  differences  between  the  schoolhouse  and 
real-world  models  because  the  real-world  model  lagged  the  new  procedures  and  had  not 
developed  a  functional  level  (i.e.,  logistics  level,  operational  interpretations,  or  office-level 
procedures).  This  would  prove  to  be  a  most  unfortunate  and  perhaps  overwhelming  effect  of  the 
dynamic  changes  to  the  documented  requirements  process. 

IDEF  Instruction 

IDEF  instruction  was  an  essential  element  of  task  preparation  because,  at  the  time  of  task 
initiation,  IDEF3  was  an  untried,  largely  undocumented,  methodology.  IDEF3  was  developed 
by  the  Texas  A&M  KBSL  while  under  contract  to  the  government.  The  only  data  sources 
available  were  three  reports  published  as  "beta  drafts"  under  the  KBSL  contract  in  April,  May, 
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and  July  of  1990.  Consequently,  Systems  Exploration  Incorporated  contracted  Knowledge 
Based  Systems,  Inc.  (KBSI)  to  conduct  an  intensive  week-long  course  tailored  to  specific  task 
requirements  and  covering  the  following  areas: 

•  basic  IDEF3  methodology; 

•  application  of  the  methodology  to  specific  task  requirements,  to  include  sample 
models,  based  on  KfiSI's  preliminary  review  of  the  pertinent  directives; 

•  use  of  the  prototype  IDEF3  tool  to  build  the  models; 

•  abstraction  of  IDEFO  models  from  IDEF3  models  to  include  mapping  from  one  to 
another  as  a  form  of  discovery  and  validation;  and 

•  use  of  an  IDEFO  tool,  the  AIO  Professional  Version. 

Phase  II:  Modeling  the  "Schoolhouse"  Requirements  Process 

The  purpose  of  Phase  II  was  to  produce  the  schoolhouse  models  of  the  requirements 
process.  The  approach  was  to  develop  both  IDEFO  and  IDEF3  models  using  the  directives  as  the 
primary  source  material,  and  to  validate  and  verify  the  models  through  interviews  with  SMEs. 

To  facilitate  the  interviews,  a  technique  was  derived  that  used  the  model  node  tree  diagrams  to 
structure  the  content  of  the  material  to  be  covered,  and  used  a  detailed  questionnaire  to  elaborate 
on  each  activity,  its  required  inputs,  and  expected  outputs/products. 

Pretesting  the  interview  technique  was  an  essential  element  of  data  collection.  Prior  to 
interviewing  SMEs  in  the  various  organizations,  the  models,  questionnaires,  and  interview 
process  were  reviewed  by  laboratory  personnel  and  tested  using  an  AFIT  faculty  member  as  a 
representative  SME.  Both  the  laboratory  reviewer  and  the  test  subject  requested  some  changes 
but  otherwise  approved  the  general  data  collection  approach  and  material. 

This  evaluation  proved  faulty  because  the  reviewers  were  not  representative  of  the 
subject  pool.  The  reviewers  were  more  familiar  with  the  topic  than  were  the  actual  SME;  that  is, 
they  were  either  more  familiar  with  the  requirements  process  or  with  the  IDEF  presentation 
technology.  Because  of  the  dissimilarity  of  the  reviewers  to  the  SME's,  the  data  collection 
approach  was  flawed.  Furthermore,  SMEs  in  the  field  are  functionally  oriented  and  possess  little 
familiarity  with  the  overall  process.  They  focus  on  specific  areas  of  expertise  such  as 
intelligence,  logistics,  or  tactics. 
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Because  of  the  SMEs  narrow  frame  of  reference,  the  questionnaire  was  too  long  and 
somewhat  intimidating— even  though  the  SMEs  were  only  asked  to  comment  on  the  parts  of  the 
process  with  which  they  were  familiar.  Care  was  taken  during  the  interview  process  to  discern 
between  the  SMEs  knowledge  of  the  process  described  by  directives  and  regulations,  and  the 
process  as  practiced  by  field  offices.  Data  pertaining  to  actual  practices  was  used  to  construct 
the  practice  model. 


Phase  III:  Modeling  the  "Practice  "  Requirements  Process 

The  purpose  of  this  phase  was  to  develop  the  "Practice"  models  and  to  further  validate 
the  Phase  II  models.  Consequently,  this  phase  was  conducted  in  the  same  manner  as  Phase  II. 
The  results  of  the  first  round  of  interviews  concerning  actual  practices  were  incorporated  into  the 
models  and  verified  during  a  second  round  of  interviews.  During  the  second  round,  the 
interviewees  possessed  a  broader  understanding  of  the  entire  process. 

Phase  IV:  Model/Process  Analysis,  Decision  Support  System 
Recommendations  and  Final  Report 

The  final  phase  of  the  task  was  to  analyze  the  mature  models  and  to  assess  the  viability  of 
using  advanced  DSS  technology  at  critical  junctures  of  the  requirements  process. 

IV.  MODELING  METHODS 

IDEF  Methodology 

The  analysis  conducted  under  this  effort  required  the  construction  of  IDEF  models  or 
descriptions  of  the  "requirements"  process.  The  model  begins  with  the  identification  of  the 
deficiency  and/or  opportunity  (pre-Milestone  0)  and  extends  through  the  Milestone  III  decision 
to  begin  the  Production  and  Deployment  phase  of  system  acquisition.  The  DoD  publications  and 
AFRs  covering  the  generation,  validation,  and  management  of  ACAT I  weapon  system 
operational  requirements  comprise  a  substantial  volume  of  material.  The  physical  acquisition 
process  extends,  depending  on  the  complexity  and  urgency  of  need,  over  five  to  ten  years. 
Therefore,  the  rationale  for  modeling  was  to  capture  and  present  the  essence  of  a  large  amount  of 
information  in  a  more  readily  absorbable  format.  Condensing  this  data  to  a  graphical  format 
facilitates  subsequent  analysis  but  requires  that  the  modeler  become  intimately  familiar  with  the 
subject  matter. 
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IDEF  is  an  integrated  computer-aided  manufacturing  approach  to  better  communicate 
and  analyze  productivity  within  environments.  An  IDEFO  model  is  used  to  produce  a  function 
or  activity  model  that  is  a  structured  representation  of  the  major  processes  within  a 
manufacturing  system  or  environment.  The  IDEFO  model  also  presents  associated  information 
and  objects  interrelating  those  processes.  The  Automated  IDEFO  modeling  tool,  also  called  the 
AIO  Professional  Version,  was  used  to  facilitate  IDEFO  modeling. 

IDEF3  is  a  scenario-driven  process  flow  description  capture  method.  Its  goal  is  to 
provide  a  structured  method  for  expressing  the  domain  expert's  knowledge  about  how  a 
particular  system  or  organization  works.  A  scenario  may  be  (a)  a  sequence  of  activities  that 
constitute  a  particular  process  or  event,  (b)  a  set  of  situations  that  describe  a  problem  in  an 
organization  or  system,  or  (c)  a  process  description  in  a  given  setting.  IDEF3,  unlike  IDEFO, 
provides  the  capability  for  capturing  temporal  precedence  and  causality  relationships  between 
activities.  In  IDEF3  terminology,  these  activities  are  referred  to  as  Units  of  Behavior  (UOBs). 
The  IDEF3  method  enables  the  modeler  to  display  a  larger  number  of  objects  or  concepts 
associated  with  a  given  activity  or  UOB  than  the  number  permitted  by  IDEFO.  The  context  of 
the  scenario  provides  a  boundary  for  the  IDEF3  description. 

The  IDEF3  method  is  applied  in  the  following  four  steps. 

1.  Identify  the  scenarios. 

2.  Model  each  scenario  by  preparing  a  diagram  of  the  process  or  event. 

3.  Prepare  an  elaboration  consisting  of  facts  and  constraints  in  natural  language  text  to 
support  the  diagram. 

4.  Decompose  scenarios  further  and  repeat  Steps  1  through  3  for  each  scenario  produced. 

An  attempt  was  made  to  use  a  prototype  IDEF3  modeling  tool.  However,  the  prototype 
was  under  development  and  unstable,  even  for  the  purpose  of  beta  testing.  After  several 
unsuccessful  attempts,  MacDraw  and  Microsoft  Word  software  were  selected  for  the  IDEF3 
model.  These  software  packages  enabled  the  production  of  a  written  report  but  did  not  provide  a 
user-friendly  interface  for  navigating  through  the  model,  as  is  the  case  with  the  AIO  tool. 
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The  Integrated  Computer-Aided  Manufacturing  Definition  Experience 


While  DDEFO  has  been  used  extensively  by  the  government  and/or  contractors  for  some 
time,  the  advent  of  IDEF3  is  relatively  recent.  TTie  technique  was  developed  by  KBSL  under 
Government  contract  and  was  delivered  to  the  Government  in  January  of  1991.  At  the  start  of 
this  task,  there  was  no  commercially  available  software  tool  to  facilitate  IDEF3  modeling.  KBSI 
provided  a  prototype  tool  which  required  a  Symbolics  list  processing  machine.  Efforts  to  locate 
government-furnished  equipment  (GFE)  for  the  contract  period  of  performance  were 
unsuccessful.  Temporary  use  of  GFE  for  IDEF3  testing  revealed  that  the  software  was  unstable 
and  hindered  progress. 

A  benefit  of  this  effort  was  the  opportunity  to  compare  IDEFO  and  IDEF3.  Since  both 
methods  model  identical  activities,  a  comparison  could  be  made  with  regard  to  technique  utility, 
descriptive  power,  and  ease  of  use  for  both  reading/understanding  and  preparing  a  model. 
However,  the  lack  of  an  IDEF3  software  tool  of  a  maturity  level  comparable  to  the  KBSI  AIO 
tool  prevented  a  direct  comparison  of  the  relative  user-friendliness  of  the  techniques  for  model 
building. 

IDEF3  offers  several  advantages.  First,  it  is  able  to  relate  activities  temporally,  that  is,  in 
the  chronological  order  in  which  they  occur.  This  is  a  major  improvement  over  IDEFO,  which 
confuses  even  experienced  IDEFO  users/readers  because  of  the  temptation  to  infer  temporal 
relationships  between  model  entities.  Although  it  may  be  erroneous  to  draw  temporal 
conclusions  for  an  IDEFO  diagram,  it  is  natural  to  assume  that  the  IDEFO  diagram  depicts 
chronological  order  from  top  left  to  bottom  right.  Users/readers  may  also  be  confused  by  the 
fact  that  within  a  single  model,  some  decompositions  accurately  reflect  temporal  relationships  by 
chance  while  other  decompositions  do  not  reflect  temporal  relationships. 

Offsetting  this  advantage  of  IDEF3,  however,  is  the  fact  that  when  modeling  a  process  as 
complex,  lengthy,  and  iterative  as  the  "requirements  process,"  the  question  of  which  parameter 
changes  first  or  which  document  is  actually  completed  first  is  irrelevant.  The  precedence  of 
changes  to  the  COEA  and  the  ORD  is  actually  immaterial.  It  is  only  essential  that  documents 
are  changed  prior  to  major  reviews  (such  as  the  Summit  and  Defense  Acquisition  Board  (DAB) 
program/milestone  reviews)  and  that  documents  are  essentially  synchronized  throughout  the 
requirements  process. 
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Another  distinction  between  IDEFO  and  IDEFS  is  that  IDEF3  does  not  impose  arbitrary 
limits  on  the  number  of  functions  on  a  page,  nor  does  it  restrict  the  number  of  objects  attached  to 
any  function  in  any  given  diagram.  Often,  in  IDEFO,  it  is  necessary  to  "bundle"  concepts  into 
abstract  groups  in  order  not  to  exceed  the  limit  of  six  concepts  per  side  of  an  IDEFO  activity,  or 
to  group  activities  into  higher-level  abstractions  so  as  not  to  exceed  the  six  activities-per-page 
limit. 


Again,  countervailing  effects  limit  this  distinction.  IDEFO  models  require  artificial 
activities  because  of  the  restrictions  on  activities  per  page.  With  the  IDEF3  technique,  some 
higher-level  artificial  activities  and  displayed  decompositions  could  be  eliminated;  however, 
these  focus  attention  on  "the  big  picture,"  which  might  be  lost  if  one  went  directly  to  the  lowest 
level  of  indenture. 

Under  this  effort,  both  the  directives  that  produced  the  "schoolhouse"  models  and  the 
interviews  with  the  SMEs  naturally  progressed  from  a  general  view/overview  to  the  more 
particular  decomposition  of  a  task.  Therefore,  it  was  logical  to  build  the  models  from  a 
hierarchical  perspective;  consequently,  there  is  a  very  close  relationship  between  IDEFO 
activities  and  IDEF3  functions. 

Notably,  the  domains  of  the  two  models  are  exactly  the  same:  the  activities  and 
responsibilities  of  various  Air  Force  personnel  involved  in  the  identification,  validation,  and 
management  of  weapon  system  requirements.  Since  the  purpose  of  IDEFO  and  rDEF3  modeling 
is  to  capture  and/or  describe  the  subject  at  hand  with  as  much  detail  and  accuracy  as  the 
techniques  allow,  substantial  similarities  between  the  IDEFO  and  IDEF3  models  are  expected. 

V.  ANALYSIS  OF  MODELS 
Assumptions  Underlying  the  Models 

The  models  depict  the  requirements/acquisition  processes  as  prescribed  in  the  pertinent 
directives  at  the  time  of  this  research.  Additionally,  they  capture  the  views  of  the  personnel 
interviewed.  However,  the  research  was  predicated  on  certain  basic  assumptions  which  seemed 
reasonable  during  task  formulation  but,  in  light  of  the  tumultuous  changes  in  both  the  acquisition 
processes  and  the  restructuring  of  the  Air  Force,  require  re-examination. 
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An  assumption  basic  to  the  task  of  modeling  the  schoolhouse  process  was  that  there 
would  be  current  directives  from  which  the  information  could  be  extracted.  This  proved  to  be  an 
invalid  assumption  since  the  primary  Air  Force  directive,  AFR  57-1,  was  not  released  in  final 
form  until  after  the  modeling  process  was  complete.  Therefore,  the  models  are  in  concert  with 
the  information  available  at  the  time. 

Two  other  rather  basic  assumptions  were  also  impacted  by  the  events  which  took  place 
during  this  research.  The  first  involves  the  SMEs’  level  of  expertise  with  regard  to  the 
schoolhouse  procedures.  For  example,  during  this  effort  three  versions  of  AFR  57-1  were  in 
use.  The  first  was  the  existing  version  dated  7  October  1988.  This  version  was  clearly 
incompatible  with  the  new  DoD  5000  series  publications.  During  most  of  the  research,  the 
SMEs  were  in  the  process  of  reviewing  and  commenting  on  the  draft  version  of  the  new 
AFR  57-1.  SMEs'  comments  and  questions  revealed  disagreement  with  and  uncertainty  over  the 
new  procedures.  The  final  version  (dated  8  November  1991)  did,  in  fact,  differ  from  the  draft. 
Therefore,  no  one  at  Air  Staff;  the  operating,  implementing,  and  supporting  commands;  or  AFIT 
could  accurately  define  the  "existing"  requirements  process. 

Second,  the  pending  reorganization  also  impacted  SME  expertise  in  regard  to  actual 
practice.  SAC,  for  example,  was  very  cooperative,  had  established  a  very  comprehensive 
requirements  process,  and  seemed  to  embrace  the  new  philosophy/procedures.  However,  SAC 
personnel  expressed  the  seeming  futility  of  documenting  the  SAC  view,  since  SAC  was  about  to 
be  deactivated  on  1  June  1992.  SAC  personnel  were  uncertain  if  the  new  Air  Combat  Command 
would  employ  SAC  methods,  employ  TAC  methods,  or  develop  a  hybrid  or  new  approach. 

Scope  of  the  Modeling  Effort 

The  scope  of  the  modeling  effort  can  best  be  termed  ambitious  in  terms  of  the  period  of 
time  modeled,  the  number  of  people/organizations  involved,  and  the  number  of  functions 
defined.  The  period  of  time  modeled  (from  identification  of  the  need  and/or  opportunity 
through  the  Milestone  HI  decision  to  begin  production)  may  be  more  than  12  years.  The  number 
of  people  involved  and/or  interviewed  includes  members  of  DoD  agencies  (e.g.,  the  JROC,  Cost 
Analysis  Improvement  Group,  DAB  Committees,  and  the  DAB  itself),  members  of  two 
operating  command  headquarters/staff  agencies,  and  AFIT  professors.  The  number  of  functions 
defined  range  from  120  in  the  objective  view  (or  schoolhouse  model)  to  87  in  the  TAC  view, 
with  93  and  105  in  the  SAC  and  ASD  models,  respectively.  Eight  models  were  produced:  a 
process  model  and  an  activity  model  (IDEF3  and  IDEFO,  respectively)  of  the  schoolhouse 
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process  (a  total  of  two),  and  process  and  activity  models  of  the  real-world  requirements 
environment  from  the  SAC,  TAG,  and  ASD  perspectives  (a  total  of  six). 

Inherent  Limitations  of  the  Modeling  Effort 

The  major  limitation  of  this  effort  was  the  relatively  small  size  of  the  total  Air  Force 
requirements  community  that  was  used  to  represent  the  entire  Air  Force.  For  example,  research 
was  limited  to  two  operating  commands,  neither  of  which  is  a  joint  combatant  command  (such  as 
Air  Force  Central  Command  which  might  have  provided  more  realistic  operational  or  combat- 
related  considerations).  Also,  only  a  small  percentage  of  the  personnel  from  each  command 
involved  were  interviewed.  Thus  the  samplings  may  not  have  been  truly  representative  of  the 
commands.  In  addition,  the  rank  and  position  of  the  personnel  were  limited;  consequently, 
firsthand  information  was  not  obtained  relative  to  the  more  senior  level  activities  (such  as 
Summit  reviews,  JROC  reviews,  DAB  Documentation  and  Committee  Reviews,  and  the  DAB 
Milestone  Review).  The  personnel  interviewed  for  this  effort  could  only  conjecture  as  to  the 
nature  of  these  activities. 

Differences  Between  the  Schoolhouse  and  Real-World  Process 

Given  the  broad  scope  of  the  requirements  process  and  the  formality  of  senior-level 
review  procedures,  the  requirements  process  as  practiced  varies  only  slightly  from  the  process  as 
prescribed.  There  are  essentially  no  major  differences  between  the  processes.  Further,  no 
deviation  from  the  final  prescribed  documents  are  permitted.  Major  commands  are  still 
formulating  unique  implementation  procedures  that  culminate  in  the  final  documents.  For 
example,  at  the  time  of  our  interviews,  TAC  had  not  yet  embraced  the  need  for  establishing  a 
CAG  (a  new  requirement  contained  in  the  draft  AFR  57-1)  and  had  no  immediate  plans  to 
convene  one  for  their  next  ACAT I  weapon  system.  However,  by  the  time  the  next  ACAT I 
weapon  system  successfully  passes  the  Milestone  0  decision  to  enter  the  Concept  Exploration 
phase,  TAC  will  likely  have  adopted  the  CAG  concept.  If  the  SAC  perspective  prevails,  the 
process  will  likely  go  one  step  beyond  required  procedures.  For  example,  SAC  has  developed  a 
preliminary  phase  in  MNS  development:  development  of  a  Tentative  Need  Statement  (TNS). 
The  TNS  is  reviewed  by  the  SAC  Requirements  Review  Group,  then  forwarded  to  the 
implementing  and  supporting  commands  as  well  as  Air  Staff  for  their  coordination  and  planning. 
TTie  TNS  becomes  a  precursor  to  the  MNS. 
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One  difference  between  the  schoolhouse  and  real-world  process  lies  in  how  some 
programs  are  initiated.  The  regulations  depict  a  process  wherein  operating  commands  identify 
needs  and/or  opportunities  through  a  variety  of  means,  largely  through  the  somewhat  vague  or 
poorly  defined  processes  known  as  MAA  and  MNA.  These  analyses  result  in  an  MNS  which  is 
validated  by  the  JROC  and,  if  favorably  reviewed  by  the  DAB,  passes  the  first  acquisition 
milestone  (Milestone  0)  and  enters  into  the  Concept  Exploration  phase  to  determine  the  most 
promising  and  satisfying  solution  to  the  identified  deficiency/opportunity.  According  to  research 
performed  by  Captain  Tom  Miller  (SAC)  and  documented  in  an  Air  Force  survey  of  operating 
conunands  including  SAC,  TAC,  MAC,  and  Air  Training  Command  (ATC),  none  of  their 
current  or  most  recent  major  programs  went  through  an  acquisition  Concept  Exploration  phase. 
Instead,  all  of  these  commands'  major  programs  were  directed  from  the  top  down. 

Opportunities  for  Decision  Support  System  Implementation 

Computer  usage  within  the  Air  Force  has  changed  dramatically  over  the  last  several 
years.  Where  once  there  were  only  a  few  in  an  office  for  people  to  share,  computers  are  now 
ubiquitous,  with  one  virtually  on  every  desk  at  the  working  level.  At  the 
time  of  this  effort.  Air  Staff  was  developing  a  data  base  to  track  information  on  acquisition 
programs  by  Program  Management  Directive  number.  Likewise,  the  major  commands  were 
tracking  the  status  of  their  requirements  packages  through  locally  developed  data  bases  by  TNS 
or  MNS. 

Decision  Support  System  Tasks 

Certain  DSS  opportunities  were  assumed  at  the  start  of  the  interview  process.  A  DSS 
was  expected  to  facilitate  decisions  regarding  trade-offs  within  operational  and  functional  areas 
as  well  as  trade-offs  between  functional  and  operational  area  requirements.  Operational  mission 
areas  are  those  defined  by  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  such  as 
Strategic  Offense,  Strategic  Defense,  Tactical  Air  Warfare,  etc.  (see  AFR  57-1,  Attachment  10). 
Functional  areas  are  those  which  affect  the  suitability  of  a  system  for  its  intended  use  such  as 
availability,  maintainability,  reliability,  logistics  supportability,  etc.  A  prospective  DSS 
environment  might  encompass  all  activities  associated  with  initial  requirements  determination, 
requirements  analysis,  and  requirements  management. 

Interviewees  were  specifically  asked  what  type  of  computer  support  would  provide  the 
most  benefit.  Notably,  some  functions  we  might  have  assumed  would  be  helpful  were  never 
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requested.  None  of  the  interviewees  requested  a  DSS  which  would  perform  a  particular  type  of 
computation,  or  apply  a  technical  or  quantitative  algorithm  to  a  specific  problem  or  trade-off 
opportunity.  There  may  be  reasons  for  this:  prospective  users  believe  the  complexity  of  these 
types  of  computations  are  best  left  to  professional  government  or  contractor  operational 
researcher  analysts;  or,  since  weapon  system  types,  current  budget  considerations,  and/or  world 
political  situations  vary  so  greatly  from  major  acquisition  to  major  acquisition,  similar  studies 
are  uncommon  and  difficult  to  generalize  as  subjects  for  prospective  analysis  tools.  Features 
unanimously  requested,  however,  were  ones  that  would  infuse  discipline  into  the  process, 
integrate  the  various  activities  and  supporting  documentation,  and  provide  generic  as  well  as 
system-specific  training. 

Mission  Area  Assessment  and  Mission  Need  Analysis 

The  first  opportunity  for  employing  a  DSS  presents  itself  in  the  areas  of  MA  A  and  MNA. 
There  is  a  need  for  a  structured,  systematic  approach  to  these  activities.  This  need  was 
documented  in  the  literature  and  verified  in  the  interviews  conducted  for  this  effort.  Kent  (1989) 
observed,  "Congress  is  growing  increasingly  critical  of  the  apparent  lack  of  a  logical  and 
persuasive  relationship  between  U.S.  military  strategies  and  the  defense  budgets  that  it  is  asked 
to  approve."  Research  at  TAC  revealed  a  TAC  study  which  recommended  that  action  officers  be 
provided  with  training  and  tools  for  performing  an  MNA  and  that  TAC  develop  a  structured 
process  to  integrate  MAA  and  MNA  findings  into  the  requirements  development  process.  SAC 
also  expressed  a  need  for  improving  the  process  and  had  initiated  a  request  for  a  Productivity, 
Reliability,  Availability,  and  Maintainability  project  to  help  them  structure  their  requirements 
process. 

A  DSS  to  support  identification  of  valid  needs  and/or  cost-effective  opportunities  is 
clearly  indicated  by  our  research  and  analysis.  The  DSS  envisioned  should  apply  a  structured 
system-engineering-oriented  approach  to  the  MAA/MNA  or  strategy-to-task  and  task-to-need 
processes.  The  DSS  could  employ  a  QFD  grid  or  matrix  to  establish  relationships  between  a 
hierarchy  of  National  Security  Objectives  (NSOs)  and  a  corresponding  hierarchy  of  military 
tasks  or  operations  as  a  first  step  in  the  process.  These  relationships  may  range  from  strongly 
positive  to  strongly  negative  and  may  depend  on  the  particular  concept  of  operations  or  relative 
importance  of  the  NSO  in  the  analysis/decision  at  hand.  Target  values  for  each  military  task 
would  also  be  established.  Computerization  of  the  process,  (i.e.,  the  ability  to  rapidly  create 
different  matrices  or  insert  different  values  to  analyze  the  robustness  of  various  concepts  of 
operation),  will  greatly  enhance  the  MAA/MNA  phase  of  the  requirements  process.  The 
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contribution  of  the  DSS  to  the  analysis  of  various  concepts  of  operation  (and  the  possible 
avoidance  of  a  materiel  solution)  may  be  of  equal  or  greater  value  than  its  contribution  to  the 
determination  of  precise  performance  operational  requirements. 

Computerization  also  facilitates  the  attachment  of  attributes  or  relational  entities  to  each 
data  element  in  the  decision  process  and  thus  creates  the  ability  to  track,  manipulate,  analyze, 
and  integrate  related  decision  parameters.  Notably,  the  DSS  employed  in  the  MAA/MNA 
processes  supports  not  only  the  "traditional"  identification  of  operational  deficiencies  and 
technological  opportunities,  but  can  also  be  used  effectively  to  support  the  development  of 
concepts  of  operation  and  mission  relationships  for  directed  systems. 

Design  Specifications 

The  power  of  the  QFD  technique  is  that  it  provides  a  framework  for  decisions  related  to 
subsequent  phases  of  the  acquisition,  (i.e.,  the  Concept  Exploration,  and  Engineering  and 
Manufacturing  Development  Phases).  The  DSS  will  support  definition  and  refinement  of  the 
requirements  during  the  Need  Determination  (pre-Milestone  0)  and  Concept  Exploration  Phases. 
Toward  the  end  of  the  Concept  Exploration  Phase,  when  the  ORD  and  Concept  Baseline  are 
developed  and  finalized,  the  set  of  operational  tasks  first  identified  during  pre-Milestone  0 
activities  and  refined  during  Phase  0  will  serve  as  the  vertical  left-hand  axis  or  starting  point  for 
a  design-specification  QFD  grid.  The  top  or  horizontal  axis  of  the  grid  will  be  the  design 
characteristics  associated  with  each  operational  requirement.  As  in  the  previous  matrix,  strong, 
medium,  weak,  or  negative  relationships  will  be  established  in  appropriate  grid  intersections.  In 
this  case,  relationships  between  design  characteristics  must  also  be  assigned.  This  is 
accomplished  in  the  "roof  of  the  matrix.  For  example,  a  design  characteristic  involving  weight 
may  have  a  positive  proportional  reiauonship  with  an  operational  requirement  for  survivability 
and  an  inversely  proportional  relationship  with  an  operational  requirement  for  range.  Since  the 
operational  requirements  are  always  traceable  back  to  the  MAA/MNA  phase,  design  trade-offs 
can  be  based  largely  on  the  "voice  of  the  customer"  rather  than  the  "voice  of  the  engineer." 
Similarly,  design  specifications  can  be  compared  to  manufacturing  requirements.  When 
manufacturing  decisions  are  weighed  against  performance  characteristics,  the  impact  of  the 
design  decision  on  mission  requirements  can  quickly  be  assessed  and  used  as  a  factor  in  the 
decision  process.  As  in  the  MAA/MNA  mode,  each  matrix  element  and/or  subelement  can  be 
embedded  with  attributes  linking  it  to  other  decisions,  rationale,  lessons  learned,  and 
documentation. 
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Integration/Project  Management 


The  acquisition  of  an  ACAT I  weapon  system  is,  of  necessity,  a  very  complex  process 
requiring  input  and  action  from  a  number  of  different  organizations  and  creating  a  myriad  of 
associated  documentation.  The  ACAT  I  acquisition  process  spawns  numerous  studies  and 
analyses,  creates  innumerable  decision/trade-off  opportunities,  and  aggregates  lessons  learned 
files  while  losing  or  failing  to  capture  decision  rationale  files/related  data.  Without  exception, 
interviewee  expressed  a  need  for  a  DSS  with  strong  data  tracking,  data  integration,  and  project 
management  capabilities. 

The  DSS  data  base  provides  audit  trail  capability  as  well  as  data  search  and  retrieval 
functions.  The  need  for  this  capability  surfaced  in  the  development  and  preparation  of  a  recent 
COEA.  One  interviewee  stated  that  he  was  vaguely  aware  of  a  previous  program-related  study 
that  would  have  been  of  great  value  to  the  COEA  effort.  However,  his  numerous  other  activities 
and  limited  time  precluded  him  from  expending  the  necessary  time  to  locate  the  study. 
Consequently,  the  COEA  was  prepared  without  benefit  of  those  study  results.  The  final  results 
of  the  COEA  may  have  been  the  same;  however,  considerable  unnecessary  time,  manpower,  and 
money  was  expended. 

In  addition,  the  DSS  will  have  the  ability  to  access,  integrate,  manipulate,  array,  and 
display  data  in  a  manner  conducive  to  producing  the  decisions  required  of  the  various  factions  of 
the  requirements/acquisition  community.  At  any  time  throughout  the  acquisition  cycle,  it  will  be 
possible  to  trace  a  specific  performance,  design,  or  manufacturing  data  element  either  back  to  its 
origin  in  the  NSOs  or  laterally  to  any  document  which  uses  it  or  produces  related  information. 
The  DSS  will  also  enforce  phase  and  configuration  control.  Users  will  be  able  to  access,  for 
analytical  purposes,  system  requirements  as  they  evolve  from  one  phase  or  document  to  another. 
Again,  for  analytical  purposes,  any  DSS  user  will  be  able  to  review  the  ORD  in  its  original  draft 
form,  as  it  appeared  prior  to  and  after  the  Summit  review,  as  it  appeared  before  and  after  DAB 
documentation  and  committee  reviews,  and  in  its  final  form  as  amended  by  the  Milestone 
review. 


Of  great  concern  to  the  Air  Force  acquisition  personnel  that  were  interviewed  was  the 
need  to  ensure  all  affected  parties  were  operating  with  the  same  data,  assumptions,  and  rationale. 
Since  there  are  so  many  different  documents  prepared  by  so  many  different  organizations 
containing  either  the  same  data  or  dependent/derived  data,  maintaining  data 
consistency/currency  is  extremely  difficult.  One  of  the  greatest  benefits  of  the  DSS  is  that  it 
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enforces  data  consistency.  The  DSS,  by  maintaining  consistency  between  documents, 
contributes  to  the  maintenance  of  discipline  among  the  various  tasks;  all  participants  will  know 
where  they  are  in  the  acquisition  cycle  and  will  be  working  the  appropriate  pieces  in  turn. 

TTie  need  for  data  consistency  is  paramount.  For  example,  the  performance  requirements 
specified  in  the  ORD  must  satisfy  the  mission  need  as  expressed  in  the  MNS.  This  set  of 
requirements  must  also  be  one  alternative  evaluated  in  the  COEA.  The  requirements  must  also 
be  identical  to  the  performance  requirements  used  to  assess  the  threat  against  the  system  in 
preparation  of  the  STAR.  The  performance  requirements  also  form  the  basis  for  determining 
both  the  independent  and  program  life  cycle  cost  estimates.  This  same  set  of  requirements  is 
also  expressed  in  the  Acquisition  Program  Baseline  (APB)  for  the  phase  in  progress.  The  APB, 
in  turn,  affects  and/or  is  directly  affected  by  the  approved  acquisition  strategy.  These  same 
requirements  are  used  as  the  basis  for  testing  procedures/criteria  in  the  preparation  of  the  TEMP. 
Progress  toward  meeting  the  same  system  performance  requirements  is  documented  in  the  SMM. 
Finally,  the  SMM  is  used  in  conjunction  with  the  IPS,  Annex  D,  Risk  Assessment,  to  identify 
risk  and  formulate  risk  reduction  plans.  The  DSS  will  identify  the  ramifications  which  changes 
to  any  one  of  the  documents  will  have  on  the  others  and  thus  facilitate  trade-off  decisions  and/or 
improve  program  management.  The  interviewees  noted  that  changes  to  the  acquisition  strategy 
and/or  funding  profile  of  acquisition  programs  occurred  frequently,  precipitating  many 
unplanned  trade-off  decisions.  A  DSS  to  support  sound  decision-making  in  such  an  unstable 
financial  environment  is  urgently  needed. 

Training 

A  primary  purpose  of  this  study  was  to  identify  opportunities  wherein  a  computerized 
DSS  could  be  used  to  improve  the  requirements  process  with  particular  emphasis  on  advancing 
the  CE  principles.  One  method  of  ensuring  that  CE  principles  are  factored  into 
requirements/acquisition-related  decision-making  is  to  embody  them  into  the  algorithms  of  a 
DSS  of  the  type  described  in  this  report.  This  type  of  DSS  may  be  readily  adapted  to  provide  an 
excellent  training  capability.  Its  use  as  a  decision/job-aiding  device  in  real  time  and  as  a  training 
vehicle  for  either  on-the-job  or  classroom  training  would  greatly  increase  the  benefits  to  be 
derived  fix)m  the  application  of  CE  principles.  Without  exception,  the  SMEs  who  participated  in 
this  research  expressed  the  need  for  a  system  that  would  help  them  in  their  job  and  provide 
training. 
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The  DSS  would  be  capable  of  providing  two  types  of  training.  The  first  is  procedural; 
that  is,  it  would  impart  knowledge  relative  to  the  DoD,  Air  Force,  and  Command- specific 
requirements/acquisition  processes,  (both  the  schoolhouse  and  real-world  procedures  modeled  in 
this  task)  with  emphasis  on  CE  considerations.  The  trainer  could  provide  initial 
lequirements/acquisition  training  to  meet  the  needs  of  those  who  are  transferred  into  acquisition 
positions  without  benefit  of  adequate  training.  The  interviewees  noted  that  inadequate  training  is 
common.  The  DSS  could  also  provide  refresher/advanced  training  to  experienced  users.  A 
procedural  training  system  is  particularly  apropos  at  this  time  because  of  the  magnitude  of  the 
recent  major  changes  to  the  DoD  directives/processes  and  their  effect  on  Air  Force  and  major 
command  implementing  directives.  These  changes  have  created  a  universal  training 
requirement;  there  is  now  as  much  need  to  train  seasoned  veterans  as  there  is  to  train  newcomers. 
Compounding  the  situation  is  the  Air  Force  reorganization  which  will  affect  the  operating, 
implementing,  and  supporting  commands  as  well  as  Air  Staff.  The  reorganization  creates  new 
commands  with  combined/modified  mission  responsibilities,  thereby  creating  an  associated  need 
for  training.  The  time,  therefore,  could  not  be  more  opportune  to  utilize  the  power  of  a  decision 
support  tool  to  focus  on  the  new  procedures/missions,  etc.-especially  one  that  promotes  the 
application  of  CE  principles.  In  regard  to  the  advancement  of  CE  as  a  factor  in 
requirements/acquisition  decisions,  the  merger  of  AFSC  and  AFLC  and  the  institution  of  the 
IWSM  concept  will  also  serve  as  a  catalyst  to  bring  about  increased  CE  awareness. 

The  second  type  of  training  is  weapon-system-specific  and  would  be  of  particular  value 
to  personnel  as  they  transferred  into  system-related  assignments.  Each  type  of  weapon  system 
(e.g.,  fighter,  bomber,  missile,  trainer,  etc.)  presents  a  different  set  of  requirements  and/or 
acquisition  problems,  and  even  similar  weapon  system  acquisition  programs  vary  because  of 
differing  conditions  at  the  time  of  the  acquisition,  (whether  political,  financial,  or  technological). 
The  tracking/integrating  capabilities  of  the  DSS,  therefore,  could  be  used  to  provide  system- 
specific  training  in  the  form  of  background  information,  lessons  learned,  decision  rationale  files, 
etc.  This  capability  alleviates  the  training  burden  associated  with  either  high  turnover  (few,  if 
any,  personnel  remain  associated  with  a  particular  system  throughout  its  entire  procurement 
cycle)  or  inadequate  initial  or  specialty  training. 

Evolutionary  Requirements  Definition 

The  purpose  of  an  Evolutionary  Requirements  Definition  (ERD)  is  to  ensure  a  logical 
progression  from  documented  operational  deficiencies/technological  opportunities  and  new 
required  capabilities  (as  expressed  in  broad  operational  terms  in  the  MNS)  to 
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manufacturing/production  quality  system  specifications  (as  expressed  in  contractual  terms  with 
the  manufacturer)  following  a  successful  Milestone  III  decision.  ERD  is  fully  described  in 
Part  4,  Section  B  of  the  new  (23  Feb  91)  DoD  Instruction  5000.2  and  reflects  the  essence  of  the 
revised  acquisition  procedures.  At  the  heart  of  the  new  philosophy  is  the  objective  of  ERD, 
which  is  to  progressively  refine  in  greater  detail  and  number  at  successive  milestone  decision 
points  the  initial  broad  objectives  and  minimum  acceptable  requirements  established  at 
Milestone  I.  ERD  is  intended  to  (a)  keep  all  reasonable  options  open  and  facilitate  cost- 
schedule-performance  trade-offs  early  in  the  process  and  (b)  avoid  premature  commitment  to  a 
system-specific  solution.  (Note  that  the  new  procedures  do  not  establish  an  SPO  until  after 
Milestone  I  vice  Milestone  0  under  the  previous  guidance.) 

One  main  benefit  of  employing  a  DSS  is  improved  quality  of  the  initial  requirements  set; 
however,  it  is  important  to  recognize  that  the  system  requirements  developed  during  the  CE 
phase  are  still  a  long  way  from  being  contractual  product  specifications.  An  indication  of  the 
"looseness"  of  the  requirements  at  this  time  is  given  by  the  APR  57-1  instructions  for  preparing 
the  initial  COEA.  A  COEA  is  a  document  initially  prepared  during  the  Concept  Exploration 
Phase  and  is  required  to  support  a  Milestone  I  decision.  Subsequent  iterations  of  the  COEA  are 
prepared  for  a  Milestone  II  decision  (and  updated  for  Milestones  III  and  IV,  if  necessary).  AFR 
57-1  acknowledges  that  the  limited  data  available  during  the  Concept  Exploration  Phase  may 
produce  only  gross  estimates  of  investment  (procurement)  costs  and  that  the  organizational  and 
operational  cost  estimates  may  only  be  rough  estimates.  AFR  57-1  further  stipulates  that  cost 
estimates  should  be  qualified,  when  necessary,  to  highlight  their  weaknesses  and  any 
possibilities  for  gross  errors. 

The  DoDI  5000.2  section  on  ERDs  also  places  increased  emphasis  on  the  involvement  of 
the  operating  command  during  the  early  phases  of  system  acquisition,  and  their  role  and 
responsibilities  in  developing  the  ORD.  The  ORD  is  the  primary  requirements  document  and  is 
the  basis  for  development  of  the  draft  system  specifications.  It  also  contains  the  minimum 
acceptable  requirements  for  key  parameters  which  are  incorporated  into  the  APBs  (Concept, 
Development,  and  Production)  and  the  TEMP  as  thresholds. 

Strategy-to-Task  Analysis 

The  main  difference  between  the  "old"  (Oct  88)  and  the  "new"  ( Nov  91)  AFR  57-1  is 
that  the  former  version  dealt  primarily  with  documentation  and  format,  (i.e.,  administrative 
details),  whereas  the  new  version  stresses  a  level  of  discipline  absent  in  the  earlier  version. 
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Because  of  the  criticality  of  these  topics,  additional  information  would  be  welcome  in  future 
editions  of  the  regulation.  The  strategy-to-task  analysis  should  be  the  critical  initial  step  in  the 
formulation  of  new  system  requirements.  Kent  further  defines  the  term  "requirements", 
identifying  legitimate  uses  and  uses  he  finds  corrupted  or  obsolete. 

For  many  years,  the  strategy-to-task  analysis,  a  pan  of  the  MAA  process,  was  either 
improperly  done,  or  not  done  at  all.  One  interviewee  reported  that  then-current  major  programs 
in  SAC,  TAC,  MAC,  and  ATC  were  directed  programs  and  were  not  subjects  of  classical 
MAA/MNA/MNS  progressions  of  need  analysis.  In  the  course  of  acquisition,  however,  these 
systems  would  still  benefit  from  a  rigorous  strategy-to-task  analysis  to  help  guide  the  many 
trade-off  decisions  that  usually  arise  for  any  potential  new  system.  Therefore,  the  development 
of  a  DSS  to  establish  unbreakable  links  back  to  the  original  or  underlying  "requirements"  in  the 
Defense  Planning  Guidance,  would  greatly  enhance  the  quality  and  integrity  of  the  requirements 
process. 

While  the  strategy-to-task  analysis  is  part  of  the  MAA  and  may  be  considered  the  logical 
starting  point  for  any  given  acquisition,  it  is  in  fact  part  of  a  continuum  that  includes  the  MNA 
and  the  whole  ERD  cycle.  A  strategy-to-task  analysis  cannot  stand  by  itself;  it  requires  a 
subsequent  task-to-need  or  MNA  to  in  order  to  have  any  relevance.  The  task-to-need  analysis 
requires  a  (or  multiple)  concept  of  operations  to  achieve  meaningful  results.  As  the 
requirements  evolve  through  the  various  stages  of  the  acquisition  process,  they  are  altered  for  a 
number  of  reasons,  for  example,  the  dynamics  of  the  design/manufacturing  interface,  changing 
threat/enemy  capability,  test  results,  etc.  Each  decision  resulting  in  a  change  to  the  design  which 
affects  performance,  regardless  of  the  cause(s),  must  be  evaluated  relative  to  its  impact  on  the 
underlying  NSOs  and,  therefore,  causes  the  MNA  and  MAA  tasks  to  be  revisited.  A  DSS  can 
facilitate  such  an  analysis  by  capturing  the  relationships  between  NSOs,  military  tasks,  and 
design/manufacturing  trade-offs  for  any  given  concept  of  operations  and  keeping  these 
relationships  at  the  forefront  of  the  decision  process.  Again,  this  reiterates  the  primary  benefit  of 
a  DSS  to  support  the  MAA/MNA  processes.  It  never  loses  sight  of  or  fails  to  inteiject  the 
original  voice  of  the  customer,  that  is,  the  NSOs. 

Emphasis  on  Early  Acquisition  Cycle  Activities 

The  new  AFR  57-1  contains  more  process  guidance  and  rationale  than  did  the  previous 
edition.  Most  of  these  changes  affect  activities  that  occur  or  are  initiated  early  in  the  acquisition 
cycle.  To  complement  the  emphasis  placed  on  MAA/MNA,  the  Air  Force  has  assigned 
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responsibility  for  developing  and  preparing  the  COEA  to  the  operating  command  (vice  the 
program  office/implementing  command).  This  is  logical  given  the  overriding  objective  of 
defining  the  "mission"  requirements  independent  of  contractual  and  budgetary  constraints 
imposed  by  commitment  to  a  system-specific  solution  too  early  in  the  process.  A  DSS  should 
prove  extremely  useful  in  developing  operational  effectiveness  measurement  criteria  to  be  used 
in  the  COEA.  To  assist  the  operating  command  in  managing  COEA  development,  conducting 
concept  studies,  and  preparing  any  other  pre-Milestone  1  documents,  the  Air  Force  has 
implemented  the  CAG  approach.  The  CAG,  created  along  with  the  publication  of  the  new  AFR 
57-1,  will  be  chaired  by  the  operating  command  (usually  the  Director  of  Requirements). 
Membership  is  determined  by  the  operating  command  but  will  usually  consist  of  the  lead 
operating  command  staff  (Deputy  Chiefs  of  Staff  for  Requirements,  Operations,  Plans  and 
Programs,  Comptroller,  Intelligence,  Logistics,  Communications-Computer  Systems,  Surgeon 
General,  Personnel,  and  Security,  as  appropriate),  the  implementing  and  supporting  command 
requirements  organizations,  the  AFOTEC,  and  other  Air  Staff  and  Secretariat  offices,  as 
appropriate.  The  DSS  should  prove  a  boon  to  the  CAG  concept  as  well,  since  it  will  greatly 
enhance  communication  among  all  members  and  ensure  that  all  members  are  current  with  regard 
to  the  latest  program/document  status. 

VI.  CONCLUSIONS 
Limitations  of  the  Study 

The  primary  limitation  of  this  effort  is  the  scope  of  the  activities  modeled  and  analyzed. 
The  acquisition  of  an  ACAT I  weapon  system  is  an  extremely  complex  and  lengthy  process.  To 
model  and  analyze  every  activity  from  the  perspective  or  viewpoint  of  the  individual  involved  is 
beyond  the  capability  of  the  resources  available  for  this  project.  Therefore,  only  the  top  level  of 
the  most  important  tasks  could  be  accurately  described. 

The  second  most  important  limitation  is  the  composition  of  the  SME  interviewee  pool. 
Given  the  scope,  the  number  of  personnel  interviewed  was  relatively  small  and,  in  all  cases, 
relatively  far  removed  from  the  upper  management  echelon  so  important  to  an  ACAT  I  program. 
For  example,  much  of  the  models  deals  with  activities  chaired  by  such  luminaries  as  the 
Assistant  Secretary  of  Defense  for  Acquisition  (the  DAB  Milestone  reviews),  the  Vice-Chairman 
of  the  Joint  Chiefs  of  Staff  (JROC  reviews),  the  CSAF  (Summits),  etc.  None  of  the  interviewees 
were  able  to  provide  much  first-hand  knowledge  of  these  activities.  Therefore,  most  of  the 
information  on  these  important  activities  was  extracted  from  the  directives.  In  addition. 
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although  the  breadth  of  activities  modeled  was  wide,  interviews  were  relatively  brief  because 
interviewees  could  spare  little  time  away  from  regular  duties. 


A  third  major  limitation  was  imposed  by  the  release  of  new  guidance  that  substantially 
changed  ACAT I  procedures  for  the  acquisition  phase  of  interest  in  this  study:  the  need 
determination  or  pre-Milestone  0  phase.  As  a  result  of  this  release,  there  were  no  cunent  Air 
Force  implementing  directives.  A  draft  version  of  AFR  57-1  was  in  use;  however,  there  were 
many  questions  and  much  uncertainty  as  to  what  the  final  version  of  the  regulation  would 
contain.  This  rendered  the  SMEs  less  than  expert  in  the  new  acquisition  procedures. 

Finally,  the  study  took  place  during  a  time  of  unprecedented  organizational  upheaval. 
Preparations  were  being  made  to  consolidate  the  AFSC  (the  implementing  command  of  the 
models)  and  the  AFLC  (the  supporting  command  of  our  models)  into  the  AFMC.  The  new 
major  command  will  begin  operations  1  July  1992.  In  addition  to  the  merger,  many  of  the 
AFSC  program  management  activities,  (primarily  those  dealing  with  the  upper/senior  levels  of 
management  responsibility  such  as  Air  Force  representation  to  the  DAB  Committee  reviews,  the 
JROC  reviews,  and  the  DAB  Milestone  reviews)  critical  to  an  ACAT  I  acquisition  had  already 
transferred  to  the  Office  of  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition  and  his  body 
of  PEOs.  Consequently,  interviewees  were  even  further  removed  from  the  workings  of  top-level 
program  management.  Furthermore,  the  operating  commands  (SAC  and  TAC)  were  also  in  the 
throes  of  reorganization/consolidation  into  the  Air  Combat  Command  (which  becomes  effective 
1  June  1992).  Reorganizations  of  this  magnitude  caused  most  SMEs  to  be  unsure  of  the  exact 
procedures  the  new  commands  would  adopt. 

General  Findings 

The  objectives  of  the  task-to  gain  a  deep  thorough  understanding  of  and  describe  the 
environment  of  the  potential  requirements-process  DSS  user  and  to  identify  opportunities  for 
which  a  DSS  could  be  beneficially  employed  were  accomplished.  A  DSS  can  fill  a  void  in  the 
current  requirements  process  by  providing  a  means  to  enhance  the  quality  of  strategy-to-task 
analyses  which  are  part  of  the  MAA  process,  by  similarly  enhancing  the  task-to-need  or  MNA 
process,  and  by  establishing  links  between  the  weapon  system  requirements  as  they  evolve 
during  the  course  of  the  acquisition  process  and  the  basic  national  security  requirements.  Since 
one  of  the  greatest  opportunities  for  DSS  employment  lies  in  the  formulation  of  traceable 
relationships  between  NSOs  and  design  considerations,  its  use  should  not  be  limited  to  pre- 
Milestone  0  activities.  These  relationships  can  be  used  to  guide  decisions  throughout  the  full 


31 


range  of  acquisition  phases/milestone  decisions.  Therefore,  the  DSS  must  be  compatible  with 
the  Program  Manager's  Support  Systems  (PMSS),  the  Acquisition  Program  Tracking  System 
(APTS),  and  other  requirements,  acquisition,  program  tracking,  and  mnagement  systems. 

The  new  directives  (in  particular  APR  57-1)  contain  much  more  rationale  and  process 
informadon  than  did  their  predecessors;  however,  the  DoD  and  Air  Force  acquisition  processes 
are  still  heavily  laden  with  documentation  requirements.  One  of  the  more  important  pre- 
Milestone  review  activities  is  the  DAB  Committee  documentation  review.  Thus,  it  follows  that 
one  of  the  major  benefits  of  the  DSS  will  be  its  ability  to  integrate  data  among  the  various 
documents.  It  is  critical  that  all  offices  with  primary  responsibilities  for  the  various  documents 
operate  with  current,  complete,  and  accurate  information.  It  is  also  imperative  that  the  decision¬ 
maker  consider  the  effect  the  decision  will  have  on  the  myriad  of  other  documents/acquisition 
processes  which  may  be  affected.  The  ability  to  integrate  and  provide  this  data  was  one  of  the 
SMEs'  most  often  requested  features  in  a  requirements  process  DSS. 

The  SMEs  believe  that  a  DSS  which  could  provide  both  process  and  system- specific 
training  would  be  of  tremendous  value  in  enhancing  the  requirements  process.  This  was 
particularly  true  of  personnel  who  were  newly  assigned  to  a  requirements/acquisition  position 
from  another  career  field,  as  is  often  the  case  with  aircrew/rated  personnel.  These  people  arc 
highly  skilled  and  possess  exceptional  practical  knowledge  as  to  what  the  system  requirements 
should  be,  but  they  often  have  no  working  knowledge  of  the  requirements/acquisition  process. 

A  DSS  which  could  impart  the  required  process  information  in  addition  to  bringing  the  newly 
assigned  person  up-to-speed  on  a  particular  system  in  acquisition  would  pay  for  itself  quickly. 
Use  of  the  DSS  would  standardize  the  process  across  all  organizations.  In  light  of  the 
organizational  changes  currently  taking  place  in  the  Air  Force,  a  standardized  training  approach 
seems  particularly  attractive. 

One  task  of  this  study  was  to  identify  the  differences  between  the  requirements  process  as 
prescribed  by  directive  and  taught  at  AFIT  (the  schoolhouse  process)  and  the  real-world  process. 
Although  the  schoolhouse  process  changed  during  the  course  of  the  study,  the  two  processes 
(i.e.,  schoolhouse  and  real-world)  are  not  now  (nor  were  they  ever)  very  different.  The  only 
differences  recorded  in  the  models  were  the  relatively  minor  SAC  variation  of  producing  a  TNS 
prior  to  a  more  formal  and  "required"  MNS,  and  the  reluctance  of  TAC  to  convene  a  CAG 
should  the  need  arise  (however,  this  may  change).  It  will  take  some  time  before  all  the  nuances 
of  the  new  procedures  are  fully  understood  and  incorporated  into  actual  practice.  Other 
differences  such  as  the  TAC  MAA  (labeled  "Review  AF  Planning  Guidance"  in  the  model)  and 
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Preliminary  Integrated  Manpower  Personnel  and  Consolidated  Training  Program  Plan  activities 
^incorporated  in  the  TAC  ORD  preparation  activities)  can  be  primarily  attributed  to  the  lack  of 
definitive  Air  Force  guidance  (which,  in  turn,  is  attributable  to  the  revised  DoD  policies)  and  to 
the  turmoil  created  by  the  complete  reorganization/dissolution  of  the  major  operating, 
implementing,  and  supporting  commands  (i.e.,  SAC,  TAC,  MAC,  AFLC,  and  AFSC). 

Future  Directions  for  Similar  Work 

Future  research  should  be  directed  toward  developing  methods  to  adapt  the  QFD  matrix 
format  to  a  more  user-friendly  personal  computer  screen-type  format.  The  result  might  resemble 
a  spreadsheet;  however,  each  cell  would  be  linked  via  attributes  to  a  relational  data  base. 

IDEFlx  is  recommended  to  develop  the  "to  be"  relational  data  base  system. 

Furthermore,  the  development  of  a  prototype  DSS  for  more  immediate  use  on 
ACAT  III/IV  acquisitions  is  recommended.  This  approach  would  facilitate  access  to  more 
"users"  of  the  system  at  relatively  higher  levels  of  authority  within  the  program  than  is  available 
in  ACAT  I/II  programs.  Also,  the  experience  gained  could  be  directly  applied  to  major 
(ACAT  l/II)  systems  in  follow-on  efforts.  Another  approach  would  be  to  develop  the  DSS  to 
satisfy  the  needs  of  major  acquisition  subsystems. 

Additional  research  is  needed  to  examine  existing  commercial  off-the-shelf  software 
applications  which  could  be  readily  adapted  to  the  DSS.  Some  of  these  applications  may  be 
sufficiently  sophisticated  to  meet  the  needs  of  ACAT  Ill/lV  systems.  For  example,  during  this 
limited  research  effort,  existing  software  packages  that  facilitate  automated  document  generation 
(such  as  system  specifications,  work  breakdown  structures,  functional  flow  diagrams,  requests 
for  proposals,  SOWs,  etc.)  were  identified.  Future  research  should  also  examine  relevant 
government  software  applications,  including  those  that  are  weapon  system  specific  but  could  be 
generalized. 
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ACRONYMS 


AIO 

ACAT 

AFTT 

AFLC 

AFMC 

AFOTEC 

AFPEO 

APR 

AFSC 

AL/HRG 

APB 

APTS 

ASD 

ATC 

CAG 

CE 

COD 

COEA 

CSAF 

DAB 

DoD 

DoDD 

DoDI 

DoDM 

DRP 

DSS 

ERD 

FY 

GFE 

HQ 

ICE 

IDEF 

IDEFO 

IDEF3 


AIO  Professional  Version  Software 

Acquisition  Category 

Air  Force  Institute  of  Technology 

Air  Force  Logistics  Command 

Air  Force  Materiel  Command 

Air  Force  Operational  Test  and  Evaluation  Center 

Air  Force  Program  Executive  Office 

Air  Force  Regulation 

Air  Force  Systems  Command 

Armstrong  Laboratory,  Logistics  Research  Division 

Acquisition  Program  Baseline 

Acquisition  Program  Tracking  System 

Aeronautical  Systems  Division  of  Air  Force  Systems  Command 

Air  Training  Command 

Concept  Action  Group 

Concurrent  Engineering 

Cooperative  Opportunities  Document 

Cost  and  Operational  Effectiveness  Analysis 

Air  Force  Chief  of  Staff 

Defense  Acquisition  Board 

Department  of  Defense 

Department  of  Defense  Directive 

Department  of  Defense  Instruction 

Department  of  Defense  Manual 

Detailed  Research  Plan 

Decision  Support  System 

Evolutionary  Requirements  Definition 

Fiscal  Year 

Government  Furnished  Equipment 
Headquarters 

Independent  Cost  Estimate 

Integrated  Computer-Aided  Manufacturing  Definition 

Integrated  Computer-Aided  Manufacturing  Definition,  Activity  Modeling 

Integrated  Computer-Aided  Manufacturing  Definition,  Process  Modeling 
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ACRONYMS  (Continued) 


IPS 

Integrated  Program  Summary 

IWSM 

Integrated  Weapon  Systems  Management 

JROC 

Joint  Requirements  Oversight  Council 

KBSI 

Knowledge  Based  Systems  Incorporated 

KBSL 

Knowledge  Based  Systems  Laboratory 

LRIP 

Low  Rate  Initial  Production 

MAA 

Mission  Area  Assessment 

MAC 

Military  Airlift  Command 

MER 

Manpower  Estimate  Report 

MNA 

Mission  Need  Analysis 

MNS 

Mission  Need  Statement 

NSO 

National  Security  Objective 

ORD 

Operational  Requirements  Document 

PEO 

Program  Executive  Officer 

PMSS 

Program  Manager's  Support  System 

QFD 

Quality  Function  Deployment 

SAC 

Strategic  Air  Command 

SAF/AQ 

Assistant  Secretary  of  the  Air  Force  (Acquisition) 

SAF/AQX 

Deputy  Assistant  Secretary  of  the  Air  Force  (Management  Policy  and  Program 
Integration) 

SME 

Subject  Matter  Expert 

SMM 

System  Maturity  Matrix 

SON 

Statement  of  Operational  Need 

SORD 

System  Operational  Requirements  Document 

SOW 

Statement  of  Work 

SPO 

System  Program  Office 

STAR 

System  Threat  Assessment  Report 

TAC 

Tactical  Air  Command 

TEMP 

Test  and  Evaluation  Master  Plan 

TNS 

Tentative  Need  Statement 

TQM 

Total  Quality  Management 

UOB 

Unit  Of  Behavior 

1S2220 
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